notebook.community

Edit and run

This notebook investigates alexnet-based model with normalisation and a new learning rate schedule.

The changes made include: increasing the number of epochs on which learning rate and momentum saturate to 250 (instead of original 25); making scaling factors of the learning rate smaller (shrink_amt = 0.99 instead of 0.9 and grow_amt = 1.01 instead 1.1); and monitoring valid_y_nll. Both best and most recent models are saved.

Quite soon, valid_y_nll started looking pretty good. Let's check the score. But first, we want to know what to compare it to. Looking at the equivalent model with original learning rate schedule (alexnet_based_norm_global.pkl):





In [1]:



    


cd ..



    


















    






/afs/inf.ed.ac.uk/user/s13/s1320903/Neuroglycerin/neukrill-net-work

















In [2]:



    


%run check_test_score.py -v run_settings/alexnet_based_norm_global.json



    


















    






Loading settings..
Loading model...
Loading data...
Applying normalisation: global
Finding batch size...
    chosen batch size 3089 for 1 batches
Compiling forward prop...
Making predictions...
    Batch 1 of 1
Log loss: 1.97530998748










    






Using gpu device 1: Tesla K40c

Just to make sure nothing goes wrong with reads/writes (as this model takes a lot less time per epoch), get a backup of the best model so far.

Now check score of our model:





In [3]:



    


%run check_test_score.py -v run_settings/alexnet_learning_rate.json



    


















    






Loading settings..
Loading model...
Loading data...
Applying normalisation: global
Finding batch size...
    chosen batch size 3089 for 1 batches
Compiling forward prop...
Making predictions...
    Batch 1 of 1
Log loss: 1.98909556846

Was hoping for a better score there... Check how many epochs each model saw:





In [4]:



    


%matplotlib inline



    











In [5]:



    


%run ~/Neuroglycerin/pylearn2/pylearn2/scripts/plot_monitor.py /disk/scratch/neuroglycerin/models/alexnet_based_norm_global.pkl



    


















    






generating names...
...done
set x_axis to example
A. learning_rate
B. momentum
C. total_seconds_last_epoch
D. train_h1_kernel_norms_max
E. train_h1_kernel_norms_mean
F. train_h1_kernel_norms_min
G. train_h1_max_x_max_u
H. train_h1_max_x_mean_u
I. train_h1_max_x_min_u
J. train_h1_mean_x_max_u
K. train_h1_mean_x_mean_u
L. train_h1_mean_x_min_u
M. train_h1_min_x_max_u
N. train_h1_min_x_mean_u
O. train_h1_min_x_min_u
P. train_h1_range_x_max_u
Q. train_h1_range_x_mean_u
R. train_h1_range_x_min_u
S. train_h2_kernel_norms_max
T. train_h2_kernel_norms_mean
U. train_h2_kernel_norms_min
V. train_h2_max_x_max_u
W. train_h2_max_x_mean_u
X. train_h2_max_x_min_u
Y. train_h2_mean_x_max_u
Z. train_h2_mean_x_mean_u
BA. train_h2_mean_x_min_u
BB. train_h2_min_x_max_u
BC. train_h2_min_x_mean_u
BD. train_h2_min_x_min_u
BE. train_h2_range_x_max_u
BF. train_h2_range_x_mean_u
BG. train_h2_range_x_min_u
BH. train_h3_kernel_norms_max
BI. train_h3_kernel_norms_mean
BJ. train_h3_kernel_norms_min
BK. train_h3_max_x_max_u
BL. train_h3_max_x_mean_u
BM. train_h3_max_x_min_u
BN. train_h3_mean_x_max_u
BO. train_h3_mean_x_mean_u
BP. train_h3_mean_x_min_u
BQ. train_h3_min_x_max_u
BR. train_h3_min_x_mean_u
BS. train_h3_min_x_min_u
BT. train_h3_range_x_max_u
BU. train_h3_range_x_mean_u
BV. train_h3_range_x_min_u
BW. train_h4_kernel_norms_max
BX. train_h4_kernel_norms_mean
BY. train_h4_kernel_norms_min
BZ. train_h4_max_x_max_u
CA. train_h4_max_x_mean_u
CB. train_h4_max_x_min_u
CC. train_h4_mean_x_max_u
CD. train_h4_mean_x_mean_u
CE. train_h4_mean_x_min_u
CF. train_h4_min_x_max_u
CG. train_h4_min_x_mean_u
CH. train_h4_min_x_min_u
CI. train_h4_range_x_max_u
CJ. train_h4_range_x_mean_u
CK. train_h4_range_x_min_u
CL. train_h5_col_norms_max
CM. train_h5_col_norms_mean
CN. train_h5_col_norms_min
CO. train_h5_max_x_max_u
CP. train_h5_max_x_mean_u
CQ. train_h5_max_x_min_u
CR. train_h5_mean_x_max_u
CS. train_h5_mean_x_mean_u
CT. train_h5_mean_x_min_u
CU. train_h5_min_x_max_u
CV. train_h5_min_x_mean_u
CW. train_h5_min_x_min_u
CX. train_h5_range_x_max_u
CY. train_h5_range_x_mean_u
CZ. train_h5_range_x_min_u
DA. train_h5_row_norms_max
DB. train_h5_row_norms_mean
DC. train_h5_row_norms_min
DD. train_objective
DE. train_term_0
DF. train_term_1_weight_decay
DG. train_y_col_norms_max
DH. train_y_col_norms_mean
DI. train_y_col_norms_min
DJ. train_y_max_max_class
DK. train_y_mean_max_class
DL. train_y_min_max_class
DM. train_y_misclass
DN. train_y_nll
DO. train_y_row_norms_max
DP. train_y_row_norms_mean
DQ. train_y_row_norms_min
DR. training_seconds_this_epoch
DS. valid_h1_kernel_norms_max
DT. valid_h1_kernel_norms_mean
DU. valid_h1_kernel_norms_min
DV. valid_h1_max_x_max_u
DW. valid_h1_max_x_mean_u
DX. valid_h1_max_x_min_u
DY. valid_h1_mean_x_max_u
DZ. valid_h1_mean_x_mean_u
EA. valid_h1_mean_x_min_u
EB. valid_h1_min_x_max_u
EC. valid_h1_min_x_mean_u
ED. valid_h1_min_x_min_u
EE. valid_h1_range_x_max_u
EF. valid_h1_range_x_mean_u
EG. valid_h1_range_x_min_u
EH. valid_h2_kernel_norms_max
EI. valid_h2_kernel_norms_mean
EJ. valid_h2_kernel_norms_min
EK. valid_h2_max_x_max_u
EL. valid_h2_max_x_mean_u
EM. valid_h2_max_x_min_u
EN. valid_h2_mean_x_max_u
EO. valid_h2_mean_x_mean_u
EP. valid_h2_mean_x_min_u
EQ. valid_h2_min_x_max_u
ER. valid_h2_min_x_mean_u
ES. valid_h2_min_x_min_u
ET. valid_h2_range_x_max_u
EU. valid_h2_range_x_mean_u
EV. valid_h2_range_x_min_u
EW. valid_h3_kernel_norms_max
EX. valid_h3_kernel_norms_mean
EY. valid_h3_kernel_norms_min
EZ. valid_h3_max_x_max_u
FA. valid_h3_max_x_mean_u
FB. valid_h3_max_x_min_u
FC. valid_h3_mean_x_max_u
FD. valid_h3_mean_x_mean_u
FE. valid_h3_mean_x_min_u
FF. valid_h3_min_x_max_u
FG. valid_h3_min_x_mean_u
FH. valid_h3_min_x_min_u
FI. valid_h3_range_x_max_u
FJ. valid_h3_range_x_mean_u
FK. valid_h3_range_x_min_u
FL. valid_h4_kernel_norms_max
FM. valid_h4_kernel_norms_mean
FN. valid_h4_kernel_norms_min
FO. valid_h4_max_x_max_u
FP. valid_h4_max_x_mean_u
FQ. valid_h4_max_x_min_u
FR. valid_h4_mean_x_max_u
FS. valid_h4_mean_x_mean_u
FT. valid_h4_mean_x_min_u
FU. valid_h4_min_x_max_u
FV. valid_h4_min_x_mean_u
FW. valid_h4_min_x_min_u
FX. valid_h4_range_x_max_u
FY. valid_h4_range_x_mean_u
FZ. valid_h4_range_x_min_u
GA. valid_h5_col_norms_max
GB. valid_h5_col_norms_mean
GC. valid_h5_col_norms_min
GD. valid_h5_max_x_max_u
GE. valid_h5_max_x_mean_u
GF. valid_h5_max_x_min_u
GG. valid_h5_mean_x_max_u
GH. valid_h5_mean_x_mean_u
GI. valid_h5_mean_x_min_u
GJ. valid_h5_min_x_max_u
GK. valid_h5_min_x_mean_u
GL. valid_h5_min_x_min_u
GM. valid_h5_range_x_max_u
GN. valid_h5_range_x_mean_u
GO. valid_h5_range_x_min_u
GP. valid_h5_row_norms_max
GQ. valid_h5_row_norms_mean
GR. valid_h5_row_norms_min
GS. valid_objective
GT. valid_term_0
GU. valid_term_1_weight_decay
GV. valid_y_col_norms_max
GW. valid_y_col_norms_mean
GX. valid_y_col_norms_min
GY. valid_y_max_max_class
GZ. valid_y_mean_max_class
HA. valid_y_min_max_class
HB. valid_y_misclass
HC. valid_y_nll
HD. valid_y_row_norms_max
HE. valid_y_row_norms_mean
HF. valid_y_row_norms_min

Put e, b, s or h in the list somewhere to plot epochs, batches, seconds, or hours, respectively.
Enter a list of channels to plot (example: A, C,F-G, h, <test_err>) or q to quit or o for options: HC










    

















    






set x_axis to example
A. learning_rate
B. momentum
C. total_seconds_last_epoch
D. train_h1_kernel_norms_max
E. train_h1_kernel_norms_mean
F. train_h1_kernel_norms_min
G. train_h1_max_x_max_u
H. train_h1_max_x_mean_u
I. train_h1_max_x_min_u
J. train_h1_mean_x_max_u
K. train_h1_mean_x_mean_u
L. train_h1_mean_x_min_u
M. train_h1_min_x_max_u
N. train_h1_min_x_mean_u
O. train_h1_min_x_min_u
P. train_h1_range_x_max_u
Q. train_h1_range_x_mean_u
R. train_h1_range_x_min_u
S. train_h2_kernel_norms_max
T. train_h2_kernel_norms_mean
U. train_h2_kernel_norms_min
V. train_h2_max_x_max_u
W. train_h2_max_x_mean_u
X. train_h2_max_x_min_u
Y. train_h2_mean_x_max_u
Z. train_h2_mean_x_mean_u
BA. train_h2_mean_x_min_u
BB. train_h2_min_x_max_u
BC. train_h2_min_x_mean_u
BD. train_h2_min_x_min_u
BE. train_h2_range_x_max_u
BF. train_h2_range_x_mean_u
BG. train_h2_range_x_min_u
BH. train_h3_kernel_norms_max
BI. train_h3_kernel_norms_mean
BJ. train_h3_kernel_norms_min
BK. train_h3_max_x_max_u
BL. train_h3_max_x_mean_u
BM. train_h3_max_x_min_u
BN. train_h3_mean_x_max_u
BO. train_h3_mean_x_mean_u
BP. train_h3_mean_x_min_u
BQ. train_h3_min_x_max_u
BR. train_h3_min_x_mean_u
BS. train_h3_min_x_min_u
BT. train_h3_range_x_max_u
BU. train_h3_range_x_mean_u
BV. train_h3_range_x_min_u
BW. train_h4_kernel_norms_max
BX. train_h4_kernel_norms_mean
BY. train_h4_kernel_norms_min
BZ. train_h4_max_x_max_u
CA. train_h4_max_x_mean_u
CB. train_h4_max_x_min_u
CC. train_h4_mean_x_max_u
CD. train_h4_mean_x_mean_u
CE. train_h4_mean_x_min_u
CF. train_h4_min_x_max_u
CG. train_h4_min_x_mean_u
CH. train_h4_min_x_min_u
CI. train_h4_range_x_max_u
CJ. train_h4_range_x_mean_u
CK. train_h4_range_x_min_u
CL. train_h5_col_norms_max
CM. train_h5_col_norms_mean
CN. train_h5_col_norms_min
CO. train_h5_max_x_max_u
CP. train_h5_max_x_mean_u
CQ. train_h5_max_x_min_u
CR. train_h5_mean_x_max_u
CS. train_h5_mean_x_mean_u
CT. train_h5_mean_x_min_u
CU. train_h5_min_x_max_u
CV. train_h5_min_x_mean_u
CW. train_h5_min_x_min_u
CX. train_h5_range_x_max_u
CY. train_h5_range_x_mean_u
CZ. train_h5_range_x_min_u
DA. train_h5_row_norms_max
DB. train_h5_row_norms_mean
DC. train_h5_row_norms_min
DD. train_objective
DE. train_term_0
DF. train_term_1_weight_decay
DG. train_y_col_norms_max
DH. train_y_col_norms_mean
DI. train_y_col_norms_min
DJ. train_y_max_max_class
DK. train_y_mean_max_class
DL. train_y_min_max_class
DM. train_y_misclass
DN. train_y_nll
DO. train_y_row_norms_max
DP. train_y_row_norms_mean
DQ. train_y_row_norms_min
DR. training_seconds_this_epoch
DS. valid_h1_kernel_norms_max
DT. valid_h1_kernel_norms_mean
DU. valid_h1_kernel_norms_min
DV. valid_h1_max_x_max_u
DW. valid_h1_max_x_mean_u
DX. valid_h1_max_x_min_u
DY. valid_h1_mean_x_max_u
DZ. valid_h1_mean_x_mean_u
EA. valid_h1_mean_x_min_u
EB. valid_h1_min_x_max_u
EC. valid_h1_min_x_mean_u
ED. valid_h1_min_x_min_u
EE. valid_h1_range_x_max_u
EF. valid_h1_range_x_mean_u
EG. valid_h1_range_x_min_u
EH. valid_h2_kernel_norms_max
EI. valid_h2_kernel_norms_mean
EJ. valid_h2_kernel_norms_min
EK. valid_h2_max_x_max_u
EL. valid_h2_max_x_mean_u
EM. valid_h2_max_x_min_u
EN. valid_h2_mean_x_max_u
EO. valid_h2_mean_x_mean_u
EP. valid_h2_mean_x_min_u
EQ. valid_h2_min_x_max_u
ER. valid_h2_min_x_mean_u
ES. valid_h2_min_x_min_u
ET. valid_h2_range_x_max_u
EU. valid_h2_range_x_mean_u
EV. valid_h2_range_x_min_u
EW. valid_h3_kernel_norms_max
EX. valid_h3_kernel_norms_mean
EY. valid_h3_kernel_norms_min
EZ. valid_h3_max_x_max_u
FA. valid_h3_max_x_mean_u
FB. valid_h3_max_x_min_u
FC. valid_h3_mean_x_max_u
FD. valid_h3_mean_x_mean_u
FE. valid_h3_mean_x_min_u
FF. valid_h3_min_x_max_u
FG. valid_h3_min_x_mean_u
FH. valid_h3_min_x_min_u
FI. valid_h3_range_x_max_u
FJ. valid_h3_range_x_mean_u
FK. valid_h3_range_x_min_u
FL. valid_h4_kernel_norms_max
FM. valid_h4_kernel_norms_mean
FN. valid_h4_kernel_norms_min
FO. valid_h4_max_x_max_u
FP. valid_h4_max_x_mean_u
FQ. valid_h4_max_x_min_u
FR. valid_h4_mean_x_max_u
FS. valid_h4_mean_x_mean_u
FT. valid_h4_mean_x_min_u
FU. valid_h4_min_x_max_u
FV. valid_h4_min_x_mean_u
FW. valid_h4_min_x_min_u
FX. valid_h4_range_x_max_u
FY. valid_h4_range_x_mean_u
FZ. valid_h4_range_x_min_u
GA. valid_h5_col_norms_max
GB. valid_h5_col_norms_mean
GC. valid_h5_col_norms_min
GD. valid_h5_max_x_max_u
GE. valid_h5_max_x_mean_u
GF. valid_h5_max_x_min_u
GG. valid_h5_mean_x_max_u
GH. valid_h5_mean_x_mean_u
GI. valid_h5_mean_x_min_u
GJ. valid_h5_min_x_max_u
GK. valid_h5_min_x_mean_u
GL. valid_h5_min_x_min_u
GM. valid_h5_range_x_max_u
GN. valid_h5_range_x_mean_u
GO. valid_h5_range_x_min_u
GP. valid_h5_row_norms_max
GQ. valid_h5_row_norms_mean
GR. valid_h5_row_norms_min
GS. valid_objective
GT. valid_term_0
GU. valid_term_1_weight_decay
GV. valid_y_col_norms_max
GW. valid_y_col_norms_mean
GX. valid_y_col_norms_min
GY. valid_y_max_max_class
GZ. valid_y_mean_max_class
HA. valid_y_min_max_class
HB. valid_y_misclass
HC. valid_y_nll
HD. valid_y_row_norms_max
HE. valid_y_row_norms_mean
HF. valid_y_row_norms_min

Put e, b, s or h in the list somewhere to plot epochs, batches, seconds, or hours, respectively.
Enter a list of channels to plot (example: A, C,F-G, h, <test_err>) or q to quit or o for options: q

















In [6]:



    


%run ~/Neuroglycerin/pylearn2/pylearn2/scripts/plot_monitor.py /disk/scratch/neuroglycerin/models/alexnet_learning_rate.pkl.backup



    


















    






generating names...
...done
set x_axis to example
A. learning_rate
B. momentum
C. total_seconds_last_epoch
D. train_h1_kernel_norms_max
E. train_h1_kernel_norms_mean
F. train_h1_kernel_norms_min
G. train_h1_max_x_max_u
H. train_h1_max_x_mean_u
I. train_h1_max_x_min_u
J. train_h1_mean_x_max_u
K. train_h1_mean_x_mean_u
L. train_h1_mean_x_min_u
M. train_h1_min_x_max_u
N. train_h1_min_x_mean_u
O. train_h1_min_x_min_u
P. train_h1_range_x_max_u
Q. train_h1_range_x_mean_u
R. train_h1_range_x_min_u
S. train_h2_kernel_norms_max
T. train_h2_kernel_norms_mean
U. train_h2_kernel_norms_min
V. train_h2_max_x_max_u
W. train_h2_max_x_mean_u
X. train_h2_max_x_min_u
Y. train_h2_mean_x_max_u
Z. train_h2_mean_x_mean_u
BA. train_h2_mean_x_min_u
BB. train_h2_min_x_max_u
BC. train_h2_min_x_mean_u
BD. train_h2_min_x_min_u
BE. train_h2_range_x_max_u
BF. train_h2_range_x_mean_u
BG. train_h2_range_x_min_u
BH. train_h3_kernel_norms_max
BI. train_h3_kernel_norms_mean
BJ. train_h3_kernel_norms_min
BK. train_h3_max_x_max_u
BL. train_h3_max_x_mean_u
BM. train_h3_max_x_min_u
BN. train_h3_mean_x_max_u
BO. train_h3_mean_x_mean_u
BP. train_h3_mean_x_min_u
BQ. train_h3_min_x_max_u
BR. train_h3_min_x_mean_u
BS. train_h3_min_x_min_u
BT. train_h3_range_x_max_u
BU. train_h3_range_x_mean_u
BV. train_h3_range_x_min_u
BW. train_h4_kernel_norms_max
BX. train_h4_kernel_norms_mean
BY. train_h4_kernel_norms_min
BZ. train_h4_max_x_max_u
CA. train_h4_max_x_mean_u
CB. train_h4_max_x_min_u
CC. train_h4_mean_x_max_u
CD. train_h4_mean_x_mean_u
CE. train_h4_mean_x_min_u
CF. train_h4_min_x_max_u
CG. train_h4_min_x_mean_u
CH. train_h4_min_x_min_u
CI. train_h4_range_x_max_u
CJ. train_h4_range_x_mean_u
CK. train_h4_range_x_min_u
CL. train_h5_col_norms_max
CM. train_h5_col_norms_mean
CN. train_h5_col_norms_min
CO. train_h5_max_x_max_u
CP. train_h5_max_x_mean_u
CQ. train_h5_max_x_min_u
CR. train_h5_mean_x_max_u
CS. train_h5_mean_x_mean_u
CT. train_h5_mean_x_min_u
CU. train_h5_min_x_max_u
CV. train_h5_min_x_mean_u
CW. train_h5_min_x_min_u
CX. train_h5_range_x_max_u
CY. train_h5_range_x_mean_u
CZ. train_h5_range_x_min_u
DA. train_h5_row_norms_max
DB. train_h5_row_norms_mean
DC. train_h5_row_norms_min
DD. train_objective
DE. train_term_0
DF. train_term_1_weight_decay
DG. train_y_col_norms_max
DH. train_y_col_norms_mean
DI. train_y_col_norms_min
DJ. train_y_max_max_class
DK. train_y_mean_max_class
DL. train_y_min_max_class
DM. train_y_misclass
DN. train_y_nll
DO. train_y_row_norms_max
DP. train_y_row_norms_mean
DQ. train_y_row_norms_min
DR. training_seconds_this_epoch
DS. valid_h1_kernel_norms_max
DT. valid_h1_kernel_norms_mean
DU. valid_h1_kernel_norms_min
DV. valid_h1_max_x_max_u
DW. valid_h1_max_x_mean_u
DX. valid_h1_max_x_min_u
DY. valid_h1_mean_x_max_u
DZ. valid_h1_mean_x_mean_u
EA. valid_h1_mean_x_min_u
EB. valid_h1_min_x_max_u
EC. valid_h1_min_x_mean_u
ED. valid_h1_min_x_min_u
EE. valid_h1_range_x_max_u
EF. valid_h1_range_x_mean_u
EG. valid_h1_range_x_min_u
EH. valid_h2_kernel_norms_max
EI. valid_h2_kernel_norms_mean
EJ. valid_h2_kernel_norms_min
EK. valid_h2_max_x_max_u
EL. valid_h2_max_x_mean_u
EM. valid_h2_max_x_min_u
EN. valid_h2_mean_x_max_u
EO. valid_h2_mean_x_mean_u
EP. valid_h2_mean_x_min_u
EQ. valid_h2_min_x_max_u
ER. valid_h2_min_x_mean_u
ES. valid_h2_min_x_min_u
ET. valid_h2_range_x_max_u
EU. valid_h2_range_x_mean_u
EV. valid_h2_range_x_min_u
EW. valid_h3_kernel_norms_max
EX. valid_h3_kernel_norms_mean
EY. valid_h3_kernel_norms_min
EZ. valid_h3_max_x_max_u
FA. valid_h3_max_x_mean_u
FB. valid_h3_max_x_min_u
FC. valid_h3_mean_x_max_u
FD. valid_h3_mean_x_mean_u
FE. valid_h3_mean_x_min_u
FF. valid_h3_min_x_max_u
FG. valid_h3_min_x_mean_u
FH. valid_h3_min_x_min_u
FI. valid_h3_range_x_max_u
FJ. valid_h3_range_x_mean_u
FK. valid_h3_range_x_min_u
FL. valid_h4_kernel_norms_max
FM. valid_h4_kernel_norms_mean
FN. valid_h4_kernel_norms_min
FO. valid_h4_max_x_max_u
FP. valid_h4_max_x_mean_u
FQ. valid_h4_max_x_min_u
FR. valid_h4_mean_x_max_u
FS. valid_h4_mean_x_mean_u
FT. valid_h4_mean_x_min_u
FU. valid_h4_min_x_max_u
FV. valid_h4_min_x_mean_u
FW. valid_h4_min_x_min_u
FX. valid_h4_range_x_max_u
FY. valid_h4_range_x_mean_u
FZ. valid_h4_range_x_min_u
GA. valid_h5_col_norms_max
GB. valid_h5_col_norms_mean
GC. valid_h5_col_norms_min
GD. valid_h5_max_x_max_u
GE. valid_h5_max_x_mean_u
GF. valid_h5_max_x_min_u
GG. valid_h5_mean_x_max_u
GH. valid_h5_mean_x_mean_u
GI. valid_h5_mean_x_min_u
GJ. valid_h5_min_x_max_u
GK. valid_h5_min_x_mean_u
GL. valid_h5_min_x_min_u
GM. valid_h5_range_x_max_u
GN. valid_h5_range_x_mean_u
GO. valid_h5_range_x_min_u
GP. valid_h5_row_norms_max
GQ. valid_h5_row_norms_mean
GR. valid_h5_row_norms_min
GS. valid_objective
GT. valid_term_0
GU. valid_term_1_weight_decay
GV. valid_y_col_norms_max
GW. valid_y_col_norms_mean
GX. valid_y_col_norms_min
GY. valid_y_max_max_class
GZ. valid_y_mean_max_class
HA. valid_y_min_max_class
HB. valid_y_misclass
HC. valid_y_nll
HD. valid_y_row_norms_max
HE. valid_y_row_norms_mean
HF. valid_y_row_norms_min

Put e, b, s or h in the list somewhere to plot epochs, batches, seconds, or hours, respectively.
Enter a list of channels to plot (example: A, C,F-G, h, <test_err>) or q to quit or o for options: HC










    

















    






set x_axis to example
A. learning_rate
B. momentum
C. total_seconds_last_epoch
D. train_h1_kernel_norms_max
E. train_h1_kernel_norms_mean
F. train_h1_kernel_norms_min
G. train_h1_max_x_max_u
H. train_h1_max_x_mean_u
I. train_h1_max_x_min_u
J. train_h1_mean_x_max_u
K. train_h1_mean_x_mean_u
L. train_h1_mean_x_min_u
M. train_h1_min_x_max_u
N. train_h1_min_x_mean_u
O. train_h1_min_x_min_u
P. train_h1_range_x_max_u
Q. train_h1_range_x_mean_u
R. train_h1_range_x_min_u
S. train_h2_kernel_norms_max
T. train_h2_kernel_norms_mean
U. train_h2_kernel_norms_min
V. train_h2_max_x_max_u
W. train_h2_max_x_mean_u
X. train_h2_max_x_min_u
Y. train_h2_mean_x_max_u
Z. train_h2_mean_x_mean_u
BA. train_h2_mean_x_min_u
BB. train_h2_min_x_max_u
BC. train_h2_min_x_mean_u
BD. train_h2_min_x_min_u
BE. train_h2_range_x_max_u
BF. train_h2_range_x_mean_u
BG. train_h2_range_x_min_u
BH. train_h3_kernel_norms_max
BI. train_h3_kernel_norms_mean
BJ. train_h3_kernel_norms_min
BK. train_h3_max_x_max_u
BL. train_h3_max_x_mean_u
BM. train_h3_max_x_min_u
BN. train_h3_mean_x_max_u
BO. train_h3_mean_x_mean_u
BP. train_h3_mean_x_min_u
BQ. train_h3_min_x_max_u
BR. train_h3_min_x_mean_u
BS. train_h3_min_x_min_u
BT. train_h3_range_x_max_u
BU. train_h3_range_x_mean_u
BV. train_h3_range_x_min_u
BW. train_h4_kernel_norms_max
BX. train_h4_kernel_norms_mean
BY. train_h4_kernel_norms_min
BZ. train_h4_max_x_max_u
CA. train_h4_max_x_mean_u
CB. train_h4_max_x_min_u
CC. train_h4_mean_x_max_u
CD. train_h4_mean_x_mean_u
CE. train_h4_mean_x_min_u
CF. train_h4_min_x_max_u
CG. train_h4_min_x_mean_u
CH. train_h4_min_x_min_u
CI. train_h4_range_x_max_u
CJ. train_h4_range_x_mean_u
CK. train_h4_range_x_min_u
CL. train_h5_col_norms_max
CM. train_h5_col_norms_mean
CN. train_h5_col_norms_min
CO. train_h5_max_x_max_u
CP. train_h5_max_x_mean_u
CQ. train_h5_max_x_min_u
CR. train_h5_mean_x_max_u
CS. train_h5_mean_x_mean_u
CT. train_h5_mean_x_min_u
CU. train_h5_min_x_max_u
CV. train_h5_min_x_mean_u
CW. train_h5_min_x_min_u
CX. train_h5_range_x_max_u
CY. train_h5_range_x_mean_u
CZ. train_h5_range_x_min_u
DA. train_h5_row_norms_max
DB. train_h5_row_norms_mean
DC. train_h5_row_norms_min
DD. train_objective
DE. train_term_0
DF. train_term_1_weight_decay
DG. train_y_col_norms_max
DH. train_y_col_norms_mean
DI. train_y_col_norms_min
DJ. train_y_max_max_class
DK. train_y_mean_max_class
DL. train_y_min_max_class
DM. train_y_misclass
DN. train_y_nll
DO. train_y_row_norms_max
DP. train_y_row_norms_mean
DQ. train_y_row_norms_min
DR. training_seconds_this_epoch
DS. valid_h1_kernel_norms_max
DT. valid_h1_kernel_norms_mean
DU. valid_h1_kernel_norms_min
DV. valid_h1_max_x_max_u
DW. valid_h1_max_x_mean_u
DX. valid_h1_max_x_min_u
DY. valid_h1_mean_x_max_u
DZ. valid_h1_mean_x_mean_u
EA. valid_h1_mean_x_min_u
EB. valid_h1_min_x_max_u
EC. valid_h1_min_x_mean_u
ED. valid_h1_min_x_min_u
EE. valid_h1_range_x_max_u
EF. valid_h1_range_x_mean_u
EG. valid_h1_range_x_min_u
EH. valid_h2_kernel_norms_max
EI. valid_h2_kernel_norms_mean
EJ. valid_h2_kernel_norms_min
EK. valid_h2_max_x_max_u
EL. valid_h2_max_x_mean_u
EM. valid_h2_max_x_min_u
EN. valid_h2_mean_x_max_u
EO. valid_h2_mean_x_mean_u
EP. valid_h2_mean_x_min_u
EQ. valid_h2_min_x_max_u
ER. valid_h2_min_x_mean_u
ES. valid_h2_min_x_min_u
ET. valid_h2_range_x_max_u
EU. valid_h2_range_x_mean_u
EV. valid_h2_range_x_min_u
EW. valid_h3_kernel_norms_max
EX. valid_h3_kernel_norms_mean
EY. valid_h3_kernel_norms_min
EZ. valid_h3_max_x_max_u
FA. valid_h3_max_x_mean_u
FB. valid_h3_max_x_min_u
FC. valid_h3_mean_x_max_u
FD. valid_h3_mean_x_mean_u
FE. valid_h3_mean_x_min_u
FF. valid_h3_min_x_max_u
FG. valid_h3_min_x_mean_u
FH. valid_h3_min_x_min_u
FI. valid_h3_range_x_max_u
FJ. valid_h3_range_x_mean_u
FK. valid_h3_range_x_min_u
FL. valid_h4_kernel_norms_max
FM. valid_h4_kernel_norms_mean
FN. valid_h4_kernel_norms_min
FO. valid_h4_max_x_max_u
FP. valid_h4_max_x_mean_u
FQ. valid_h4_max_x_min_u
FR. valid_h4_mean_x_max_u
FS. valid_h4_mean_x_mean_u
FT. valid_h4_mean_x_min_u
FU. valid_h4_min_x_max_u
FV. valid_h4_min_x_mean_u
FW. valid_h4_min_x_min_u
FX. valid_h4_range_x_max_u
FY. valid_h4_range_x_mean_u
FZ. valid_h4_range_x_min_u
GA. valid_h5_col_norms_max
GB. valid_h5_col_norms_mean
GC. valid_h5_col_norms_min
GD. valid_h5_max_x_max_u
GE. valid_h5_max_x_mean_u
GF. valid_h5_max_x_min_u
GG. valid_h5_mean_x_max_u
GH. valid_h5_mean_x_mean_u
GI. valid_h5_mean_x_min_u
GJ. valid_h5_min_x_max_u
GK. valid_h5_min_x_mean_u
GL. valid_h5_min_x_min_u
GM. valid_h5_range_x_max_u
GN. valid_h5_range_x_mean_u
GO. valid_h5_range_x_min_u
GP. valid_h5_row_norms_max
GQ. valid_h5_row_norms_mean
GR. valid_h5_row_norms_min
GS. valid_objective
GT. valid_term_0
GU. valid_term_1_weight_decay
GV. valid_y_col_norms_max
GW. valid_y_col_norms_mean
GX. valid_y_col_norms_min
GY. valid_y_max_max_class
GZ. valid_y_mean_max_class
HA. valid_y_min_max_class
HB. valid_y_misclass
HC. valid_y_nll
HD. valid_y_row_norms_max
HE. valid_y_row_norms_mean
HF. valid_y_row_norms_min

Put e, b, s or h in the list somewhere to plot epochs, batches, seconds, or hours, respectively.
Enter a list of channels to plot (example: A, C,F-G, h, <test_err>) or q to quit or o for options: q

Let it run a little longer.

Check again, best file first:





In [18]:



    


%run ~/Neuroglycerin/pylearn2/pylearn2/scripts/plot_monitor.py /disk/scratch/neuroglycerin/models/alexnet_learning_rate.pkl



    


















    






generating names...
...done
set x_axis to example
A. learning_rate
B. momentum
C. total_seconds_last_epoch
D. train_h1_kernel_norms_max
E. train_h1_kernel_norms_mean
F. train_h1_kernel_norms_min
G. train_h1_max_x_max_u
H. train_h1_max_x_mean_u
I. train_h1_max_x_min_u
J. train_h1_mean_x_max_u
K. train_h1_mean_x_mean_u
L. train_h1_mean_x_min_u
M. train_h1_min_x_max_u
N. train_h1_min_x_mean_u
O. train_h1_min_x_min_u
P. train_h1_range_x_max_u
Q. train_h1_range_x_mean_u
R. train_h1_range_x_min_u
S. train_h2_kernel_norms_max
T. train_h2_kernel_norms_mean
U. train_h2_kernel_norms_min
V. train_h2_max_x_max_u
W. train_h2_max_x_mean_u
X. train_h2_max_x_min_u
Y. train_h2_mean_x_max_u
Z. train_h2_mean_x_mean_u
BA. train_h2_mean_x_min_u
BB. train_h2_min_x_max_u
BC. train_h2_min_x_mean_u
BD. train_h2_min_x_min_u
BE. train_h2_range_x_max_u
BF. train_h2_range_x_mean_u
BG. train_h2_range_x_min_u
BH. train_h3_kernel_norms_max
BI. train_h3_kernel_norms_mean
BJ. train_h3_kernel_norms_min
BK. train_h3_max_x_max_u
BL. train_h3_max_x_mean_u
BM. train_h3_max_x_min_u
BN. train_h3_mean_x_max_u
BO. train_h3_mean_x_mean_u
BP. train_h3_mean_x_min_u
BQ. train_h3_min_x_max_u
BR. train_h3_min_x_mean_u
BS. train_h3_min_x_min_u
BT. train_h3_range_x_max_u
BU. train_h3_range_x_mean_u
BV. train_h3_range_x_min_u
BW. train_h4_kernel_norms_max
BX. train_h4_kernel_norms_mean
BY. train_h4_kernel_norms_min
BZ. train_h4_max_x_max_u
CA. train_h4_max_x_mean_u
CB. train_h4_max_x_min_u
CC. train_h4_mean_x_max_u
CD. train_h4_mean_x_mean_u
CE. train_h4_mean_x_min_u
CF. train_h4_min_x_max_u
CG. train_h4_min_x_mean_u
CH. train_h4_min_x_min_u
CI. train_h4_range_x_max_u
CJ. train_h4_range_x_mean_u
CK. train_h4_range_x_min_u
CL. train_h5_col_norms_max
CM. train_h5_col_norms_mean
CN. train_h5_col_norms_min
CO. train_h5_max_x_max_u
CP. train_h5_max_x_mean_u
CQ. train_h5_max_x_min_u
CR. train_h5_mean_x_max_u
CS. train_h5_mean_x_mean_u
CT. train_h5_mean_x_min_u
CU. train_h5_min_x_max_u
CV. train_h5_min_x_mean_u
CW. train_h5_min_x_min_u
CX. train_h5_range_x_max_u
CY. train_h5_range_x_mean_u
CZ. train_h5_range_x_min_u
DA. train_h5_row_norms_max
DB. train_h5_row_norms_mean
DC. train_h5_row_norms_min
DD. train_objective
DE. train_term_0
DF. train_term_1_weight_decay
DG. train_y_col_norms_max
DH. train_y_col_norms_mean
DI. train_y_col_norms_min
DJ. train_y_max_max_class
DK. train_y_mean_max_class
DL. train_y_min_max_class
DM. train_y_misclass
DN. train_y_nll
DO. train_y_row_norms_max
DP. train_y_row_norms_mean
DQ. train_y_row_norms_min
DR. training_seconds_this_epoch
DS. valid_h1_kernel_norms_max
DT. valid_h1_kernel_norms_mean
DU. valid_h1_kernel_norms_min
DV. valid_h1_max_x_max_u
DW. valid_h1_max_x_mean_u
DX. valid_h1_max_x_min_u
DY. valid_h1_mean_x_max_u
DZ. valid_h1_mean_x_mean_u
EA. valid_h1_mean_x_min_u
EB. valid_h1_min_x_max_u
EC. valid_h1_min_x_mean_u
ED. valid_h1_min_x_min_u
EE. valid_h1_range_x_max_u
EF. valid_h1_range_x_mean_u
EG. valid_h1_range_x_min_u
EH. valid_h2_kernel_norms_max
EI. valid_h2_kernel_norms_mean
EJ. valid_h2_kernel_norms_min
EK. valid_h2_max_x_max_u
EL. valid_h2_max_x_mean_u
EM. valid_h2_max_x_min_u
EN. valid_h2_mean_x_max_u
EO. valid_h2_mean_x_mean_u
EP. valid_h2_mean_x_min_u
EQ. valid_h2_min_x_max_u
ER. valid_h2_min_x_mean_u
ES. valid_h2_min_x_min_u
ET. valid_h2_range_x_max_u
EU. valid_h2_range_x_mean_u
EV. valid_h2_range_x_min_u
EW. valid_h3_kernel_norms_max
EX. valid_h3_kernel_norms_mean
EY. valid_h3_kernel_norms_min
EZ. valid_h3_max_x_max_u
FA. valid_h3_max_x_mean_u
FB. valid_h3_max_x_min_u
FC. valid_h3_mean_x_max_u
FD. valid_h3_mean_x_mean_u
FE. valid_h3_mean_x_min_u
FF. valid_h3_min_x_max_u
FG. valid_h3_min_x_mean_u
FH. valid_h3_min_x_min_u
FI. valid_h3_range_x_max_u
FJ. valid_h3_range_x_mean_u
FK. valid_h3_range_x_min_u
FL. valid_h4_kernel_norms_max
FM. valid_h4_kernel_norms_mean
FN. valid_h4_kernel_norms_min
FO. valid_h4_max_x_max_u
FP. valid_h4_max_x_mean_u
FQ. valid_h4_max_x_min_u
FR. valid_h4_mean_x_max_u
FS. valid_h4_mean_x_mean_u
FT. valid_h4_mean_x_min_u
FU. valid_h4_min_x_max_u
FV. valid_h4_min_x_mean_u
FW. valid_h4_min_x_min_u
FX. valid_h4_range_x_max_u
FY. valid_h4_range_x_mean_u
FZ. valid_h4_range_x_min_u
GA. valid_h5_col_norms_max
GB. valid_h5_col_norms_mean
GC. valid_h5_col_norms_min
GD. valid_h5_max_x_max_u
GE. valid_h5_max_x_mean_u
GF. valid_h5_max_x_min_u
GG. valid_h5_mean_x_max_u
GH. valid_h5_mean_x_mean_u
GI. valid_h5_mean_x_min_u
GJ. valid_h5_min_x_max_u
GK. valid_h5_min_x_mean_u
GL. valid_h5_min_x_min_u
GM. valid_h5_range_x_max_u
GN. valid_h5_range_x_mean_u
GO. valid_h5_range_x_min_u
GP. valid_h5_row_norms_max
GQ. valid_h5_row_norms_mean
GR. valid_h5_row_norms_min
GS. valid_objective
GT. valid_term_0
GU. valid_term_1_weight_decay
GV. valid_y_col_norms_max
GW. valid_y_col_norms_mean
GX. valid_y_col_norms_min
GY. valid_y_max_max_class
GZ. valid_y_mean_max_class
HA. valid_y_min_max_class
HB. valid_y_misclass
HC. valid_y_nll
HD. valid_y_row_norms_max
HE. valid_y_row_norms_mean
HF. valid_y_row_norms_min

Put e, b, s or h in the list somewhere to plot epochs, batches, seconds, or hours, respectively.
Enter a list of channels to plot (example: A, C,F-G, h, <test_err>) or q to quit or o for options: HC










    

















    






set x_axis to example
A. learning_rate
B. momentum
C. total_seconds_last_epoch
D. train_h1_kernel_norms_max
E. train_h1_kernel_norms_mean
F. train_h1_kernel_norms_min
G. train_h1_max_x_max_u
H. train_h1_max_x_mean_u
I. train_h1_max_x_min_u
J. train_h1_mean_x_max_u
K. train_h1_mean_x_mean_u
L. train_h1_mean_x_min_u
M. train_h1_min_x_max_u
N. train_h1_min_x_mean_u
O. train_h1_min_x_min_u
P. train_h1_range_x_max_u
Q. train_h1_range_x_mean_u
R. train_h1_range_x_min_u
S. train_h2_kernel_norms_max
T. train_h2_kernel_norms_mean
U. train_h2_kernel_norms_min
V. train_h2_max_x_max_u
W. train_h2_max_x_mean_u
X. train_h2_max_x_min_u
Y. train_h2_mean_x_max_u
Z. train_h2_mean_x_mean_u
BA. train_h2_mean_x_min_u
BB. train_h2_min_x_max_u
BC. train_h2_min_x_mean_u
BD. train_h2_min_x_min_u
BE. train_h2_range_x_max_u
BF. train_h2_range_x_mean_u
BG. train_h2_range_x_min_u
BH. train_h3_kernel_norms_max
BI. train_h3_kernel_norms_mean
BJ. train_h3_kernel_norms_min
BK. train_h3_max_x_max_u
BL. train_h3_max_x_mean_u
BM. train_h3_max_x_min_u
BN. train_h3_mean_x_max_u
BO. train_h3_mean_x_mean_u
BP. train_h3_mean_x_min_u
BQ. train_h3_min_x_max_u
BR. train_h3_min_x_mean_u
BS. train_h3_min_x_min_u
BT. train_h3_range_x_max_u
BU. train_h3_range_x_mean_u
BV. train_h3_range_x_min_u
BW. train_h4_kernel_norms_max
BX. train_h4_kernel_norms_mean
BY. train_h4_kernel_norms_min
BZ. train_h4_max_x_max_u
CA. train_h4_max_x_mean_u
CB. train_h4_max_x_min_u
CC. train_h4_mean_x_max_u
CD. train_h4_mean_x_mean_u
CE. train_h4_mean_x_min_u
CF. train_h4_min_x_max_u
CG. train_h4_min_x_mean_u
CH. train_h4_min_x_min_u
CI. train_h4_range_x_max_u
CJ. train_h4_range_x_mean_u
CK. train_h4_range_x_min_u
CL. train_h5_col_norms_max
CM. train_h5_col_norms_mean
CN. train_h5_col_norms_min
CO. train_h5_max_x_max_u
CP. train_h5_max_x_mean_u
CQ. train_h5_max_x_min_u
CR. train_h5_mean_x_max_u
CS. train_h5_mean_x_mean_u
CT. train_h5_mean_x_min_u
CU. train_h5_min_x_max_u
CV. train_h5_min_x_mean_u
CW. train_h5_min_x_min_u
CX. train_h5_range_x_max_u
CY. train_h5_range_x_mean_u
CZ. train_h5_range_x_min_u
DA. train_h5_row_norms_max
DB. train_h5_row_norms_mean
DC. train_h5_row_norms_min
DD. train_objective
DE. train_term_0
DF. train_term_1_weight_decay
DG. train_y_col_norms_max
DH. train_y_col_norms_mean
DI. train_y_col_norms_min
DJ. train_y_max_max_class
DK. train_y_mean_max_class
DL. train_y_min_max_class
DM. train_y_misclass
DN. train_y_nll
DO. train_y_row_norms_max
DP. train_y_row_norms_mean
DQ. train_y_row_norms_min
DR. training_seconds_this_epoch
DS. valid_h1_kernel_norms_max
DT. valid_h1_kernel_norms_mean
DU. valid_h1_kernel_norms_min
DV. valid_h1_max_x_max_u
DW. valid_h1_max_x_mean_u
DX. valid_h1_max_x_min_u
DY. valid_h1_mean_x_max_u
DZ. valid_h1_mean_x_mean_u
EA. valid_h1_mean_x_min_u
EB. valid_h1_min_x_max_u
EC. valid_h1_min_x_mean_u
ED. valid_h1_min_x_min_u
EE. valid_h1_range_x_max_u
EF. valid_h1_range_x_mean_u
EG. valid_h1_range_x_min_u
EH. valid_h2_kernel_norms_max
EI. valid_h2_kernel_norms_mean
EJ. valid_h2_kernel_norms_min
EK. valid_h2_max_x_max_u
EL. valid_h2_max_x_mean_u
EM. valid_h2_max_x_min_u
EN. valid_h2_mean_x_max_u
EO. valid_h2_mean_x_mean_u
EP. valid_h2_mean_x_min_u
EQ. valid_h2_min_x_max_u
ER. valid_h2_min_x_mean_u
ES. valid_h2_min_x_min_u
ET. valid_h2_range_x_max_u
EU. valid_h2_range_x_mean_u
EV. valid_h2_range_x_min_u
EW. valid_h3_kernel_norms_max
EX. valid_h3_kernel_norms_mean
EY. valid_h3_kernel_norms_min
EZ. valid_h3_max_x_max_u
FA. valid_h3_max_x_mean_u
FB. valid_h3_max_x_min_u
FC. valid_h3_mean_x_max_u
FD. valid_h3_mean_x_mean_u
FE. valid_h3_mean_x_min_u
FF. valid_h3_min_x_max_u
FG. valid_h3_min_x_mean_u
FH. valid_h3_min_x_min_u
FI. valid_h3_range_x_max_u
FJ. valid_h3_range_x_mean_u
FK. valid_h3_range_x_min_u
FL. valid_h4_kernel_norms_max
FM. valid_h4_kernel_norms_mean
FN. valid_h4_kernel_norms_min
FO. valid_h4_max_x_max_u
FP. valid_h4_max_x_mean_u
FQ. valid_h4_max_x_min_u
FR. valid_h4_mean_x_max_u
FS. valid_h4_mean_x_mean_u
FT. valid_h4_mean_x_min_u
FU. valid_h4_min_x_max_u
FV. valid_h4_min_x_mean_u
FW. valid_h4_min_x_min_u
FX. valid_h4_range_x_max_u
FY. valid_h4_range_x_mean_u
FZ. valid_h4_range_x_min_u
GA. valid_h5_col_norms_max
GB. valid_h5_col_norms_mean
GC. valid_h5_col_norms_min
GD. valid_h5_max_x_max_u
GE. valid_h5_max_x_mean_u
GF. valid_h5_max_x_min_u
GG. valid_h5_mean_x_max_u
GH. valid_h5_mean_x_mean_u
GI. valid_h5_mean_x_min_u
GJ. valid_h5_min_x_max_u
GK. valid_h5_min_x_mean_u
GL. valid_h5_min_x_min_u
GM. valid_h5_range_x_max_u
GN. valid_h5_range_x_mean_u
GO. valid_h5_range_x_min_u
GP. valid_h5_row_norms_max
GQ. valid_h5_row_norms_mean
GR. valid_h5_row_norms_min
GS. valid_objective
GT. valid_term_0
GU. valid_term_1_weight_decay
GV. valid_y_col_norms_max
GW. valid_y_col_norms_mean
GX. valid_y_col_norms_min
GY. valid_y_max_max_class
GZ. valid_y_mean_max_class
HA. valid_y_min_max_class
HB. valid_y_misclass
HC. valid_y_nll
HD. valid_y_row_norms_max
HE. valid_y_row_norms_mean
HF. valid_y_row_norms_min

Put e, b, s or h in the list somewhere to plot epochs, batches, seconds, or hours, respectively.
Enter a list of channels to plot (example: A, C,F-G, h, <test_err>) or q to quit or o for options: q

And the most recent now:





In [19]:



    


%run ~/Neuroglycerin/pylearn2/pylearn2/scripts/plot_monitor.py /disk/scratch/neuroglycerin/models/alexnet_learning_rate_recent.pkl



    


















    






generating names...
...done
set x_axis to example
A. learning_rate
B. momentum
C. total_seconds_last_epoch
D. train_h1_kernel_norms_max
E. train_h1_kernel_norms_mean
F. train_h1_kernel_norms_min
G. train_h1_max_x_max_u
H. train_h1_max_x_mean_u
I. train_h1_max_x_min_u
J. train_h1_mean_x_max_u
K. train_h1_mean_x_mean_u
L. train_h1_mean_x_min_u
M. train_h1_min_x_max_u
N. train_h1_min_x_mean_u
O. train_h1_min_x_min_u
P. train_h1_range_x_max_u
Q. train_h1_range_x_mean_u
R. train_h1_range_x_min_u
S. train_h2_kernel_norms_max
T. train_h2_kernel_norms_mean
U. train_h2_kernel_norms_min
V. train_h2_max_x_max_u
W. train_h2_max_x_mean_u
X. train_h2_max_x_min_u
Y. train_h2_mean_x_max_u
Z. train_h2_mean_x_mean_u
BA. train_h2_mean_x_min_u
BB. train_h2_min_x_max_u
BC. train_h2_min_x_mean_u
BD. train_h2_min_x_min_u
BE. train_h2_range_x_max_u
BF. train_h2_range_x_mean_u
BG. train_h2_range_x_min_u
BH. train_h3_kernel_norms_max
BI. train_h3_kernel_norms_mean
BJ. train_h3_kernel_norms_min
BK. train_h3_max_x_max_u
BL. train_h3_max_x_mean_u
BM. train_h3_max_x_min_u
BN. train_h3_mean_x_max_u
BO. train_h3_mean_x_mean_u
BP. train_h3_mean_x_min_u
BQ. train_h3_min_x_max_u
BR. train_h3_min_x_mean_u
BS. train_h3_min_x_min_u
BT. train_h3_range_x_max_u
BU. train_h3_range_x_mean_u
BV. train_h3_range_x_min_u
BW. train_h4_kernel_norms_max
BX. train_h4_kernel_norms_mean
BY. train_h4_kernel_norms_min
BZ. train_h4_max_x_max_u
CA. train_h4_max_x_mean_u
CB. train_h4_max_x_min_u
CC. train_h4_mean_x_max_u
CD. train_h4_mean_x_mean_u
CE. train_h4_mean_x_min_u
CF. train_h4_min_x_max_u
CG. train_h4_min_x_mean_u
CH. train_h4_min_x_min_u
CI. train_h4_range_x_max_u
CJ. train_h4_range_x_mean_u
CK. train_h4_range_x_min_u
CL. train_h5_col_norms_max
CM. train_h5_col_norms_mean
CN. train_h5_col_norms_min
CO. train_h5_max_x_max_u
CP. train_h5_max_x_mean_u
CQ. train_h5_max_x_min_u
CR. train_h5_mean_x_max_u
CS. train_h5_mean_x_mean_u
CT. train_h5_mean_x_min_u
CU. train_h5_min_x_max_u
CV. train_h5_min_x_mean_u
CW. train_h5_min_x_min_u
CX. train_h5_range_x_max_u
CY. train_h5_range_x_mean_u
CZ. train_h5_range_x_min_u
DA. train_h5_row_norms_max
DB. train_h5_row_norms_mean
DC. train_h5_row_norms_min
DD. train_objective
DE. train_term_0
DF. train_term_1_weight_decay
DG. train_y_col_norms_max
DH. train_y_col_norms_mean
DI. train_y_col_norms_min
DJ. train_y_max_max_class
DK. train_y_mean_max_class
DL. train_y_min_max_class
DM. train_y_misclass
DN. train_y_nll
DO. train_y_row_norms_max
DP. train_y_row_norms_mean
DQ. train_y_row_norms_min
DR. training_seconds_this_epoch
DS. valid_h1_kernel_norms_max
DT. valid_h1_kernel_norms_mean
DU. valid_h1_kernel_norms_min
DV. valid_h1_max_x_max_u
DW. valid_h1_max_x_mean_u
DX. valid_h1_max_x_min_u
DY. valid_h1_mean_x_max_u
DZ. valid_h1_mean_x_mean_u
EA. valid_h1_mean_x_min_u
EB. valid_h1_min_x_max_u
EC. valid_h1_min_x_mean_u
ED. valid_h1_min_x_min_u
EE. valid_h1_range_x_max_u
EF. valid_h1_range_x_mean_u
EG. valid_h1_range_x_min_u
EH. valid_h2_kernel_norms_max
EI. valid_h2_kernel_norms_mean
EJ. valid_h2_kernel_norms_min
EK. valid_h2_max_x_max_u
EL. valid_h2_max_x_mean_u
EM. valid_h2_max_x_min_u
EN. valid_h2_mean_x_max_u
EO. valid_h2_mean_x_mean_u
EP. valid_h2_mean_x_min_u
EQ. valid_h2_min_x_max_u
ER. valid_h2_min_x_mean_u
ES. valid_h2_min_x_min_u
ET. valid_h2_range_x_max_u
EU. valid_h2_range_x_mean_u
EV. valid_h2_range_x_min_u
EW. valid_h3_kernel_norms_max
EX. valid_h3_kernel_norms_mean
EY. valid_h3_kernel_norms_min
EZ. valid_h3_max_x_max_u
FA. valid_h3_max_x_mean_u
FB. valid_h3_max_x_min_u
FC. valid_h3_mean_x_max_u
FD. valid_h3_mean_x_mean_u
FE. valid_h3_mean_x_min_u
FF. valid_h3_min_x_max_u
FG. valid_h3_min_x_mean_u
FH. valid_h3_min_x_min_u
FI. valid_h3_range_x_max_u
FJ. valid_h3_range_x_mean_u
FK. valid_h3_range_x_min_u
FL. valid_h4_kernel_norms_max
FM. valid_h4_kernel_norms_mean
FN. valid_h4_kernel_norms_min
FO. valid_h4_max_x_max_u
FP. valid_h4_max_x_mean_u
FQ. valid_h4_max_x_min_u
FR. valid_h4_mean_x_max_u
FS. valid_h4_mean_x_mean_u
FT. valid_h4_mean_x_min_u
FU. valid_h4_min_x_max_u
FV. valid_h4_min_x_mean_u
FW. valid_h4_min_x_min_u
FX. valid_h4_range_x_max_u
FY. valid_h4_range_x_mean_u
FZ. valid_h4_range_x_min_u
GA. valid_h5_col_norms_max
GB. valid_h5_col_norms_mean
GC. valid_h5_col_norms_min
GD. valid_h5_max_x_max_u
GE. valid_h5_max_x_mean_u
GF. valid_h5_max_x_min_u
GG. valid_h5_mean_x_max_u
GH. valid_h5_mean_x_mean_u
GI. valid_h5_mean_x_min_u
GJ. valid_h5_min_x_max_u
GK. valid_h5_min_x_mean_u
GL. valid_h5_min_x_min_u
GM. valid_h5_range_x_max_u
GN. valid_h5_range_x_mean_u
GO. valid_h5_range_x_min_u
GP. valid_h5_row_norms_max
GQ. valid_h5_row_norms_mean
GR. valid_h5_row_norms_min
GS. valid_objective
GT. valid_term_0
GU. valid_term_1_weight_decay
GV. valid_y_col_norms_max
GW. valid_y_col_norms_mean
GX. valid_y_col_norms_min
GY. valid_y_max_max_class
GZ. valid_y_mean_max_class
HA. valid_y_min_max_class
HB. valid_y_misclass
HC. valid_y_nll
HD. valid_y_row_norms_max
HE. valid_y_row_norms_mean
HF. valid_y_row_norms_min

Put e, b, s or h in the list somewhere to plot epochs, batches, seconds, or hours, respectively.
Enter a list of channels to plot (example: A, C,F-G, h, <test_err>) or q to quit or o for options: HC










    

















    






set x_axis to example
A. learning_rate
B. momentum
C. total_seconds_last_epoch
D. train_h1_kernel_norms_max
E. train_h1_kernel_norms_mean
F. train_h1_kernel_norms_min
G. train_h1_max_x_max_u
H. train_h1_max_x_mean_u
I. train_h1_max_x_min_u
J. train_h1_mean_x_max_u
K. train_h1_mean_x_mean_u
L. train_h1_mean_x_min_u
M. train_h1_min_x_max_u
N. train_h1_min_x_mean_u
O. train_h1_min_x_min_u
P. train_h1_range_x_max_u
Q. train_h1_range_x_mean_u
R. train_h1_range_x_min_u
S. train_h2_kernel_norms_max
T. train_h2_kernel_norms_mean
U. train_h2_kernel_norms_min
V. train_h2_max_x_max_u
W. train_h2_max_x_mean_u
X. train_h2_max_x_min_u
Y. train_h2_mean_x_max_u
Z. train_h2_mean_x_mean_u
BA. train_h2_mean_x_min_u
BB. train_h2_min_x_max_u
BC. train_h2_min_x_mean_u
BD. train_h2_min_x_min_u
BE. train_h2_range_x_max_u
BF. train_h2_range_x_mean_u
BG. train_h2_range_x_min_u
BH. train_h3_kernel_norms_max
BI. train_h3_kernel_norms_mean
BJ. train_h3_kernel_norms_min
BK. train_h3_max_x_max_u
BL. train_h3_max_x_mean_u
BM. train_h3_max_x_min_u
BN. train_h3_mean_x_max_u
BO. train_h3_mean_x_mean_u
BP. train_h3_mean_x_min_u
BQ. train_h3_min_x_max_u
BR. train_h3_min_x_mean_u
BS. train_h3_min_x_min_u
BT. train_h3_range_x_max_u
BU. train_h3_range_x_mean_u
BV. train_h3_range_x_min_u
BW. train_h4_kernel_norms_max
BX. train_h4_kernel_norms_mean
BY. train_h4_kernel_norms_min
BZ. train_h4_max_x_max_u
CA. train_h4_max_x_mean_u
CB. train_h4_max_x_min_u
CC. train_h4_mean_x_max_u
CD. train_h4_mean_x_mean_u
CE. train_h4_mean_x_min_u
CF. train_h4_min_x_max_u
CG. train_h4_min_x_mean_u
CH. train_h4_min_x_min_u
CI. train_h4_range_x_max_u
CJ. train_h4_range_x_mean_u
CK. train_h4_range_x_min_u
CL. train_h5_col_norms_max
CM. train_h5_col_norms_mean
CN. train_h5_col_norms_min
CO. train_h5_max_x_max_u
CP. train_h5_max_x_mean_u
CQ. train_h5_max_x_min_u
CR. train_h5_mean_x_max_u
CS. train_h5_mean_x_mean_u
CT. train_h5_mean_x_min_u
CU. train_h5_min_x_max_u
CV. train_h5_min_x_mean_u
CW. train_h5_min_x_min_u
CX. train_h5_range_x_max_u
CY. train_h5_range_x_mean_u
CZ. train_h5_range_x_min_u
DA. train_h5_row_norms_max
DB. train_h5_row_norms_mean
DC. train_h5_row_norms_min
DD. train_objective
DE. train_term_0
DF. train_term_1_weight_decay
DG. train_y_col_norms_max
DH. train_y_col_norms_mean
DI. train_y_col_norms_min
DJ. train_y_max_max_class
DK. train_y_mean_max_class
DL. train_y_min_max_class
DM. train_y_misclass
DN. train_y_nll
DO. train_y_row_norms_max
DP. train_y_row_norms_mean
DQ. train_y_row_norms_min
DR. training_seconds_this_epoch
DS. valid_h1_kernel_norms_max
DT. valid_h1_kernel_norms_mean
DU. valid_h1_kernel_norms_min
DV. valid_h1_max_x_max_u
DW. valid_h1_max_x_mean_u
DX. valid_h1_max_x_min_u
DY. valid_h1_mean_x_max_u
DZ. valid_h1_mean_x_mean_u
EA. valid_h1_mean_x_min_u
EB. valid_h1_min_x_max_u
EC. valid_h1_min_x_mean_u
ED. valid_h1_min_x_min_u
EE. valid_h1_range_x_max_u
EF. valid_h1_range_x_mean_u
EG. valid_h1_range_x_min_u
EH. valid_h2_kernel_norms_max
EI. valid_h2_kernel_norms_mean
EJ. valid_h2_kernel_norms_min
EK. valid_h2_max_x_max_u
EL. valid_h2_max_x_mean_u
EM. valid_h2_max_x_min_u
EN. valid_h2_mean_x_max_u
EO. valid_h2_mean_x_mean_u
EP. valid_h2_mean_x_min_u
EQ. valid_h2_min_x_max_u
ER. valid_h2_min_x_mean_u
ES. valid_h2_min_x_min_u
ET. valid_h2_range_x_max_u
EU. valid_h2_range_x_mean_u
EV. valid_h2_range_x_min_u
EW. valid_h3_kernel_norms_max
EX. valid_h3_kernel_norms_mean
EY. valid_h3_kernel_norms_min
EZ. valid_h3_max_x_max_u
FA. valid_h3_max_x_mean_u
FB. valid_h3_max_x_min_u
FC. valid_h3_mean_x_max_u
FD. valid_h3_mean_x_mean_u
FE. valid_h3_mean_x_min_u
FF. valid_h3_min_x_max_u
FG. valid_h3_min_x_mean_u
FH. valid_h3_min_x_min_u
FI. valid_h3_range_x_max_u
FJ. valid_h3_range_x_mean_u
FK. valid_h3_range_x_min_u
FL. valid_h4_kernel_norms_max
FM. valid_h4_kernel_norms_mean
FN. valid_h4_kernel_norms_min
FO. valid_h4_max_x_max_u
FP. valid_h4_max_x_mean_u
FQ. valid_h4_max_x_min_u
FR. valid_h4_mean_x_max_u
FS. valid_h4_mean_x_mean_u
FT. valid_h4_mean_x_min_u
FU. valid_h4_min_x_max_u
FV. valid_h4_min_x_mean_u
FW. valid_h4_min_x_min_u
FX. valid_h4_range_x_max_u
FY. valid_h4_range_x_mean_u
FZ. valid_h4_range_x_min_u
GA. valid_h5_col_norms_max
GB. valid_h5_col_norms_mean
GC. valid_h5_col_norms_min
GD. valid_h5_max_x_max_u
GE. valid_h5_max_x_mean_u
GF. valid_h5_max_x_min_u
GG. valid_h5_mean_x_max_u
GH. valid_h5_mean_x_mean_u
GI. valid_h5_mean_x_min_u
GJ. valid_h5_min_x_max_u
GK. valid_h5_min_x_mean_u
GL. valid_h5_min_x_min_u
GM. valid_h5_range_x_max_u
GN. valid_h5_range_x_mean_u
GO. valid_h5_range_x_min_u
GP. valid_h5_row_norms_max
GQ. valid_h5_row_norms_mean
GR. valid_h5_row_norms_min
GS. valid_objective
GT. valid_term_0
GU. valid_term_1_weight_decay
GV. valid_y_col_norms_max
GW. valid_y_col_norms_mean
GX. valid_y_col_norms_min
GY. valid_y_max_max_class
GZ. valid_y_mean_max_class
HA. valid_y_min_max_class
HB. valid_y_misclass
HC. valid_y_nll
HD. valid_y_row_norms_max
HE. valid_y_row_norms_mean
HF. valid_y_row_norms_min

Put e, b, s or h in the list somewhere to plot epochs, batches, seconds, or hours, respectively.
Enter a list of channels to plot (example: A, C,F-G, h, <test_err>) or q to quit or o for options: q

The score it gets on the holdout set is:





In [17]:



    


%run check_test_score.py -v run_settings/alexnet_learning_rate.json



    


















    






Loading settings..
Loading model...
Loading data...
Applying normalisation: global
Finding batch size...
    chosen batch size 3089 for 1 batches
Compiling forward prop...
Making predictions...
    Batch 1 of 1
Log loss: 2.05778594131

It got worse!

Now check scores of two models with only number of saturating epoch changed. One monitored valid_y_nll, another valid_objective.





In [20]:



    


%run check_test_score.py -v run_settings/alexnet_learning_rate2.json



    


















    






Loading settings..
Loading model...
Loading data...
Applying normalisation: global
Finding batch size...
    chosen batch size 3089 for 1 batches
Compiling forward prop...
Making predictions...
    Batch 1 of 1
Log loss: 1.98382238088

















In [21]:



    


%run check_test_score.py -v run_settings/alexnet_learning_rate3.json



    


















    






Loading settings..
Loading model...
Loading data...
Applying normalisation: global
Finding batch size...
    chosen batch size 3089 for 1 batches
Compiling forward prop...
Making predictions...
    Batch 1 of 1
Log loss: 1.95892133093

The one monitoring valid_objective seems to have done a little better, so will keep this for the development of the current best model.

Content source: Neuroglycerin/neukrill-net-work

Similar notebooks:

notebook.community | gallery | about